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A Method and System for Super Resolution 

FIELD OF THE INVENTION 
The present invention relates to super resolution enhancement of 

sampled data. 

BACKGROUND OF THE INVENTION 

Super-resolution refers to the enhancement of resolutions for 
images of scenery or objects acquired by an image capture device, such as a 
scanner or a CCD video camera, beyond the resolution inherent in the capture 
device, and beyond other limitations of the capture device. Super-resolution is 
used primarily to enhance the quality of captured img^ges, and to increase the 
mfoimation content therein. 

Many super-resolution techniques used with imaging sensors are 
based on micro scanning operations. When using micro-scanning operations, a 
scene or object being captured is sampled multiple times, each tune with a sub 
pixel shift applied. 

Prior art techniques for super-resolution do not achieve exact 
super resolution, but rather they perform over-sampling. Exaa super resolution is 
limited by the fill factor, i.e., the ratio between the area of the region sensitive to 
radiation and the pitch. The pitch is tiie area between centers of adjacent detectors 
in a sensor panel - equivalendy, the pitch is the area between adjacent pixels. For 
example, if the sensitivity to radiation only extends over half the distance 
separating detectors, then the fill factor is 0 J * 0.5 = 25%. Witii a 25% fill factor, 
exact super-resolution using prior art methods can be obtained up to a factor of 
two in resolution in each dimension, since the regions of sensitivity to radiation 
are only half tiie size of die pitch, and can thus be shifted by half-pixels in each 
dimension without overlapping. Additional super-resolution, beyond a faaor of 
two, has been achieved by prior art methods at the expense of decreased contrast 

For a fill factor of 80%, which is typical for image sensing 
devices, the limitation on exact super-resolution is ^Jq^"^^'^^' ^® 

experimental barrier is approximately at a factor of 1.5, with some decrease in 
contrast. 
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Prior art methods for obtaining super resolution from captured 
images are based on estimating signal distortion. These methods perform optimal 
estimation of signal distortion from the captured images, using Bayesian 
techniques and using criteria such as Maximum Entropy. These methods are 
advantageous when image acquisition is made over a large distance (such as 
satellite data acquired from outer space), since atmospheric transmission and 
turbulence have a major impact on limiting the resolution obtained. 

One such method is described in I. Cheeseman, B. Kanefsky, R. 
Kraft, J. Stutz, R. Hanson, "Super-Resolved Surface Reconstraction from 
Multiple Images," in Maximum Entropy and Bayesian Methods, G. R. Heidbreder 
(ed.), Kluwer, the Netheriands, 1996, pgs. 293 - 308. This method is based on 
inverse graphics theory, and is used for ground modeling from outer space 
observations. Ad initial ground model is formed by letting each pixel "vote" on 
what the corresponding ground position should be, based upon the extent that the 
corresponding ground position contributes to that pixel. The initial ground model 
is then used to project what an image should be (i.e., predict each pixel value). 
The difierences between the predicted pixel values and the observed pixel values 
are used to update the ground model until it cannot be further improved. This 
procedure produces an increase in both spatial resolution and gray-scale 
resolution. 

Another such method is described m A. Zomet and S. Pdeg, 
"Applying Super-Resolution to Panoramic Mosaics," tfff Workshop on 
Applications of Computer Vision, Princeton, Oct 1998. This method attains siqier 
resolution using an iterative method for mosaicing. Given a video sequence 
scanning a static scene, a panoramic image can be constructed whose field of view 
is the entire scene. Each region in the panorama is covered by many overlapping 
image segments, and this can be exploited to enhance its resolution. 

Another such method is described in Pearson, T. J. and Readhead, 
A. C. S. "Image Formation by Self-Calibration in Radio Astronomy," Ann. Rev. 
Astron. Astrophys. 22, 1984. pgs. 97-130. This naethod uses non linear methods, 
and is based on optimal spectrum estimation. 
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SUMMARY OF THE INVENTION 



The present specification concerns data obtained by sampling a 
continuous signal, and describes methods and systems for enhancing the resolution 
of the data. For example, a barcode reader samples a barcode when scanning it, 
and the present invention can be used to enhance the resolution of the sampled 
data, thereby providing a better reconstruction of the barcode. For another 
example, a CCD camera samples an object or scene being viewed, to produce a 
digital image, and the present invention can be used to enhance the resolution of 
the digital image, thereby providing a better quality image. 

The present invention enhances sampled data by effectively 
decreasing the sampling period. When used with digital images, the present 
invention provides sub-pixel accuracy. An original image quantized into pixel 
area elements can be enhanced using the present invention to a finer granularity 
quantization with sub-pixel area elements. For example, the enhanced image can 
have one-eighth pixel granularity. 

The present invention overcomes the limitation of prior art 
methods to achieve exact resolution improvement by more than a factor of 1.5. 
Using the present mvention, any desired resolution improvement may be obtained 
by attaching an appropriately designed mask to a sensor plane of a capture device. 

There is provided in accordance with a preferred embodiment of 
the present invention a method for enhancing the resolution of an image sensing 
device, including the steps of attaching a mask to a panel of detectors in an image 
sensing device, generating multiple fields of view, the multiple fields of view 
being related to one another by sub-pixel shifts, acquiring multiple images with 
the image sensing device from the multiple fields of view, and combining the 
multiple images into an enhanced image of higher pixel resolution than the pixel 
resolutions of the multiple images. 

There is further provided in accordance with a preferred 
embodiment of the present invention a system for enhancing the resolution of an 
image sensing device, including an image sensing device comprising a panel of 
detectors, a mask attached to said panel of detectors, a motion generator 
generating multiple fields of view, the multiple fields of view being related to one 
another by sub-pixel shifts, image acquisition circuitry housed within said image 
sensing device acquiring multiple images from the multiple fields of view, and a 
combiner combining the multiple images into an enhanced image of higher pixel 
resolution than the pixel resolutions of the multiple images. 
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There is still further provided in accordance with a preferred 
embcdiment of the present invention a method for enhancing the resolution of an 
image sensing device, including the steps of creating replicas of fields of view 
using an optical element attached to an image sensing device, acquiring multiple 
images with the sensing device from the replicas of fields of view, and combining 
the multiple images into an enhanced image of higher pixel resolution than the 
pixel resolutions of the multiple images. 

There is additionally provided in accordance with a preferred 
embodiment of the present invention a system for enhancing the resolution of an 
image sensing device, including an image sensing device, an optical element 
attached to the image sensing device, the optical element being such as to create 
replicas of fields of view, image acquisition circuitry housed within the image 
sensing device acquiring multiple images from the replicas of fields of view, and a 
combiner combining the multiple images into an enhanced image of higher pixel 
resolution than the pixel resolutions of the multiple images. 
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BRIEF DESCRIp-nON OF THE DRAWINGS 



The present invention wiU be more fully understood and 
appreciated from the following detailed description, taken in conjunction with the 
drawings in which: 

Figure ! is an illustration of a one-dimensional cross-section of a 
prior art image sensing device, such as a CCD camera or a scanner; 

Figure 2 is a prior art illustration of a pixel sensitivity 
(apodization) function, for modeling averaged light intensity arriving at a detector; 

Figures 3A - 3C are illustrations of the results of computer 
simulations of a preferred embodiment of the present invention, as applied to CCD 
cameras; 

Figures 4A - 4D are further illustrations of the results of 
computer simulations of a preferred embodiment of the present invention, as 
applied to CCD cameras; 

Figures 5A • 5C are illustrations of the results of computer 
simulations of a preferred embodiment of the present invention, as applied to 
barcode readers; 

Figure 6 is an illustration of three samplings of an object by a 
sensing device, separated by sub-pixel shifts, in accordance with a preferred 
embodiment of the present invention; 

Figure 7 is a simplified illustration of a mask attached to a sensor 
panel of anay detectors in an image sensing device, in accordance with a preferred 
embodiment of the present invention; 

Figure 8 illustrates the shape of a mask in accordance with a 
preferred embodiment of the present invention; 

Figures 9A - 9J illustrate the use of a mask in obtaining super- 
resolution, in accordance with a preferred embodiment of the present invention; 

Figure 10 illustrates an enhanced image obtained by applying 
super-resolution to the low resolution CCD captured image illustrated in Figure 
3B, without the use of a mask; and 

Figure 11 is a simplified illustration of an alternating 
("checkerboard") mask attached to a sensor panel of array detectors in an image 
sensing device, in accordance with a preferred embodiment of the present 
invention. 



5 



wo 00/59206 PCT/ILOO/00201 
DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT 



The present invention concerns digital images captured by digital 
image acquisition devices, such as digital cameras and scanners, and provides a 
method and system for obtaining super-resolution imagery. Super-resolution 
refers to the ability to obtain a resolution of a digital image that is greater than that 
native resolution of the image acquisition device. The present invention uses 
multiple low resolution captures of a picture to obtain a high resolution digital 
image. For example, if 64 images of an object are captured by a CCD camera 
having a spatial resolution of 32 x 32 pixels, the present invention can be used to 
integrate the captured data and provide a digital image of the object at a spatial 
resolution of 256 x 256 pixels. The present invention makes efficient use of 
captured image data, in that there is a direct one-to-one relationship between the 
number of images captured and the ratio of resolution enhancement. That is, if if 
images are captured then the present invention enhances resolution of the capture 
device by a factor ofK. 

Additionally, the present invention applies to acquisition devices, 
regardless of their dynamic color range. In the example described above, the CCD 
camera can have as low as 1 bit per pixel color depth, or as high as 8 bits per pixel 
and higher, and the present invention is applicable. 

As used m the present specification in reference to digital images, 
the terms "low resolution" and "high resolution" are intended to be relative 
terms, used to indicate an advantage of the present invention in providing digital 
images of an input object at a higher resolution than those produced by an image 
acquisition device. 

Reference is now made to Figure I, which illustrates a one- 
dimensional cross-section of a prior art image sensing device, such as a CCD 
camera or a scanner. An image sensing device 100 consists of an array of 
detectors 110 in a sensing panel (not shown), each of which measures a light 
intensity of a scene 120 at a specific pixel location. Light rays 130 emanating 
from scene 120 are diffracted by a lens 140, and arrive at detectors 110. 

The array of detectors 110 in the sensing panel is two- 
dimensional, and the one-dimensional array of detectors Olustrated in Figure 1 is 
but a single cross-section of the two-dimensional array. Similarly, Figure 1 
illustrates a single cross-section of scene 120. The direction of the one- 
dimensional array of detectors is denoted by a variable x, and the light intensity of 
the scene is correspondingly denoted by u(x). The variable x is a continuous 
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variable, and scene 120 corresponds to a continuous signal. Tb. array of detectors 
10 samples scene 120. and produces a discrete digital image, comprising pixel 
locauons and pucel color intensities. The spacings between detectors corresponds 
to the pixel size for the sampled digital image, and is referred to as the "pitch." 

Reference is now made to Figure 2, which iUustrates a one- 
d.mensional cross-section of a prior art pixel sensitivity (apodization) function, for 
modelmg averaged light intensity arriving at a detector. n,e measured light 
.ntensuy at each detector 110 (Figure 1) is not a measure of the light intensity at a 
smgle point location of scene 120. Instead, it is an average intensity, averaged by 
a pixel sensiUvity function gfx), or apodization. The pixel sensitivity function is 
.ilasirated as a one -dimensional function g(x) in Figure 2. but g(x) is only a single 
cross-section of a two-dimensional pixel sensitivity function g(x, y). 

The support of the two-dimensional pixel sensitivity function 
corresponds to the region that is sensitive to radiation. TWs region typically 
extends across an area that is contained within the area between neighboring pixel 
centers. Hie ratio between the area of the two-dimensional support of the pixel 
sensitivity function and Uie area between pixel centers containing it is referred to 
asthe"fillfector." 

It is the averaging witii g(x) that gives rise to complications in 
obtaining exact super-resolution in prior art meUiods. 

Reference is now made to Figures 3A - 3C, which illustrate die 
results of computer simulations of a preferred embodiment of the present 
invention, as applied to CCD cameras. Figure 3A shows an input picture with 
spatial resolution of 256 x 256 pixels. Figure 3B shows an image of tht input 
picture, captured by a CCD camera having a spatial resolution of 32 x 32 pixels, 
and 8 bits per pixel color depth. Figure 3C shows tiie reconstructed image* 
obtained by the present invention by using 64 picmres like the one Ulustrated in 
Figure 3B. It can be seen that substantially all of tiie spatial details contained 
within tiie input picture are reconstructed in tiie image of Figure 3C. 

Reference is now made to Figures 4A - 4D, which further 
illustrate tiie results of computer simulations of a preferred embodiment of tiie 
present invention, as appUed to CCD cameras. Figure 4A shows a low resolution 
image captured from tiie input picture iUustrated in Figure 3A, using a CCD 
camera having a spatial resolution of 32 x 32 pixels and a color deptii of 1 bit per 
pixel. It can be seen that tiie image shown in Figure 4A is a black and white 
image. Figure 4B shows tiie reconstructed image obtained by tiie present 
invention by using 64 pictures like tiie one Ulustrated in Figure 4A. It can be seen 
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that although Figure 4B is not identical to Figure 3A, it nevertheless is a much 
better approximation than is Figure 4A. both in terns of spatial resolution and in 
terms of dynamic color range. 

Similarly, Figure 4C shows a low resolution image captured fiom 
the input picture illustrated in Figure 3A, using a CCD camera having a spatial 
resolution of 32 x 32 pixels and a color depth of 2 bits per pbcel. It can be seen 
that the image shown in Figure 4C is comprised of four colors. Figure 4D shows 
the reconstructed image obtained by the present invention, by using 64 pictures 
like the one Ulustrated in Figure 4C It can be seen tiiat Figure 4D is almost 
identical to Figure 3A. 

When the present invention was applied to a CCD camera having 
a spatial resolution of 32 x 32 pixels and a color deptii of 4 bits per pixel (not 
shown), the reconstructed image (not shown) was indistinguishable from tiie 
original. 

Reference is now made to Figures 5 A - 5C, which illustrates tiie 
results of computer simulations of a preferred embodiment of flie present 
invention, as applied to barcode readers. Shown in Figure 5A is an original 
barcode. Shown in Figure 5B is tiie image captured by a bar code reader. As a 
result of the limited resolving capabUity of tiie barcode reader, tiie captured image 
shown in Figure SB is substantiaUy different from tiie barcode shown in Figure 
5 A. Shown in Figure 5C is a reconstiiicted image of tiie barcode using flie present 
invention, based on eight vertical replications. A comparison between Figure 5A 
and Figure 5C indicates that tiie present invention successfully overcomes die 
limitations of conventional barcode readers in resolving scanned barcodes. 

The present specification desCTibes tiie process for acquiring 
multiple low resolution images from a single picnire, and tiie way to combine tiie 
multiple low resolution image data to generate a high resolution image. 

Conventions and Nntatinn 

In tiie ensuing description it is assumed tiiat a sensing device 
samples an object being acquired in lines, such as horizontal Imes, using iV pixels 
per line. The present invention is preferably applied separately in two dimensions, 
such as in a horizontal dimension and a vertical dimension, to increase tiie pixel 
resolutions in each of tiie dimensions. Since tiie applications of tiie present 
invention in each of tiie dimensions are similar, for tiie sake of clarity and 
definiteness tfie present specification describes a preferred embodiment of tiie 
present invention in a single dimension, such as a horizontal dimension. In 
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particular, rather than use two coordinates (x, y) in the discussion below, a single x 
coordinate is used, and rather than work with double integrals and double sums, 
single integrals and single sums are used. Similarly, two-dimensional doubly 
periodic masking of a two-dimensional array of detectors in a sensing panel is 
described hereinbelow using a one-dimensional periodic masking function m(x). 

When the sensing device samples a line, the detected energy of 
the /2-lh pixel is given by 

u{n\ = \u{x + n6x)g{x)dx , n=0.1, N-l , (1) 

where u(x) denotes the captured object along a specific line, /te denotes the pixel 
width and^C;c; denotes the pixel sensitivity (i.e. apodizaiion). 

Multiple^ Image Acq?ijffition 

In a preferred embodiment of the present invention, the sensing 
device samples a line K times, each time with a shift in the captured object relative 
to the capture device by an amount oi Ax I K, Letting uk[n] denote the detected 
energy of the /i-th pixel in the *-th sampling. Equation 1 generalizes to 

«*W= \u{x^n6x^k—)g{x)dx , 

n = 0,l,...,N^l; k^O,l...,K'l. (2) 
Together, the various samples u/cfnj provide KN samples given by 

y[n] = ju(x + n -j^)^ W^fe , « = ft 2, KN-l . (3) 

In an preferred embodiment of the present invention, the shifts in 
successive image acquisitions by /ix / K are implemented by using vibrations of 
the sensing device. For example, the sensmg device can be placed on a vibrating 
platform. The parameters of the vibrations are estimated by vibration sensors or 
by use of appropriate algorithms. The vibration parameters are used to 
synchronize the sampling of the object so that the sampling occurs at times when 
the vibrated sensing device is located at appropriate sub-pixel shifts. 

In an alternate embodiment of the present invention, an assembly 
of one or more rotated mirrors can be used to shift successive fields of view of a 
sensing device by sub-pixel shifts. The rotated mirrors reflect the scene to the 
sensing device. 

In yet another embodiment^ the shifts in successive image 
acquisitions by idx / if can be implemented using an optical element instead of the 
imaging lens of the sensing device. The optical element is attached to the aperture 
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Of the sensing device, and serves like a grating to create replicated views of an 
object. Typically gratings result in repUcas since the Fourier transform of a 
grating is an impulse train of delta functions. This embodiment using an optical 
element to create replicated views is particularly weU suited for objects that are 
sufficiently small so that the replicas do not overlap. 

The abovementioned optical element can be designed, for 
example, by a multi-facet lens. It can also be designed by an algorithm such as 
the one described in Z. Zalevsky, D. Mendlovic and A. W. Lohmann, Gerchberg- 
Saxton algorithm applied in the fractional Fourier or the Fresnel domain. Optical 
Letters 21, 1996, pages 842 - 844, Uie contents of which are hereby incorporated 
by reference. 

It should be apparent to tiiose skilled in the art tiiat it is not 
necessary that tiie multiple acquired Images be separated by identical sub-pixel 
shifts. The present invention applies to multiple acquired images separated by 
subi)ixel shifts of arbitrary sizes. Moreover, successive shifts between acquired 
images do not have to be sub-pixel. The shifts can be larger dian one pixel shifts, 
as long as the multiple acquired images are all non-integral pixel shifts from one 
another. Specifically, tiie relative shifts of die multiple acquired images with 
reference to a fixed origin can be set to values rgAc, riAa, rj^.i^e, as long as 
Uie differences r/ - ry are non-integral for any distinct indices i and The case of 
equal sub-pixel shifts described above corresponds to rjfe = )fc IK. 

Reference is now made to Figure 6, which illustrates three 
samplings of an object by a sensing device, separated by sub-pixel shifts, in 
accordance widi a preferred embodiment of tiie present invention. The object is 
indicated by a curve u(x), denoted by reference numeral 610. The curve v^x) 
denotes the color of the object along a line, as a function of position, x. Also 
indicated in Figure 6 is a curve gfjcj denoting tiie sensitivity of a pixel, denoted by 
reference numeral 620. The curve g(x) is a characteristic of tiie sensing device, 
and indicates how color values are localized by averaging. 

Each row in Figure 6 indicates a sampling. The first row samples 
the object uix) at positions indicated by dashed vertical lines 630. The six pixel 
values are denoted by uo[0], uo[l], ... uo[5]. The second row samples die object 
u(x) at shifted positions indicated by dashed vertical lines 640. The six pixel 
values are denoted by ui[0], ujflj, ... ujfSJ. The tiiird row samples die object 
u(x) at shifted positions indicated by dashed vertical lines 650. The six pixel 
values are denoted by U2f0j, U2[l], ... U2[5]. 
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The samplings indicated by the second and third rows is often 
referred to as "sub-pixel" sampling, since these samplings are centered at 
fractional pixel locations. Specifically, in Figure 6, the samples in the second row 
are centered at the one-third pixel locations, and the samples in the third row are 
centered at the two-thirds pLxel locations. The value of K for the geometry 
illustrated in Figure. 6 is ii: = J, The three sets of samples together forai the 
eighteen samples y[0] = uo[0], y[l] = uj/O/, y[2] = U2[0], y[3] = uo[l], 
y[17] = U2[5], form the fiiU set of samples at all of the pixel thirds locations. 

These eighteen samples y[n] are used to reconstruct the values u(0), u(^), 

2Ay 17Ax 
«C— ). «(— r— ) of object u(x). 

The present invention uses the samples yfnj to approximately 

reconstruct the values of u{n—). Determining these reconstructed values 

K 

achieves super-resolution with a resolution enhancement factor of K, since the 
sensing device has a resolution of N pixels per line, and the reconstructed signal 
has KN pixels per line. 

In order to describe the reconstruction of the values u(n—) from 

the samples yfnjy a classical result from signal processing is used, which relates 
the discrete Fourier transform of a sampled analog signal to the continuous 
Fourier transform of the analog signal. 

A Bit of History: Frequency Representation of Sampling 

In the ensuing description, s(t) is used to denote a continuous time 
signal, and 5(7/?) is used to denote its continuous-time Fourier transform, defined 
by: 

5(yn)=£^(0e"'^^. (4) 

Periodic sampling of s(t) with a sampling period of T produces a discrete time 
signal sfnj, defined by: 

sfnj = s(nT) . (5) 
The discrete-time Fourier transform of sfn] is denoted by S(ei°'), and is defined 
by: 

5(e^^)=|;5[/i]e-^- . (6) 

An age-old result, referred to as the Poisson Summation Formula 
and which is one of the most prominent formulas used in discrete signal 

11 



wo 00/59206 PCT/ILOO/00201 
processing, provides a relationship between the continuous-tinie Fourier transfonn 
S(jn) of a continuous time signal and the discrete-time Fourier transfonn 5fe/*) of 
the sampled signal. Specifically, the Poisson Summation Fonnula states that 

is(nT) = ^ts(-A. (7) 

In fact, more generally the Poisson Summation Formula establishes that 

ts(t^nT) = l±S(-j^)e-''^, (8) 

of which Equation 7 above is a special case corresponding to r = 0. 

A simple proof of Equation 8, as indicated on page 210 of Rudin, 
W., "Real and Complex Analysis: Second Edition," McGraw-HUl, 1974, is as 
follows. The function yfr; defined by 

m-ts'it^nT) (9) 



is a periodic function with period T. As such, it has a standard Fourier series 
expansion of the form 

« Ida 

/(o=£c*e'^~, (10) 

it— (O 

where the Fourier coefficient c/c is given by 

1 (T — 

^*'jf/(Oe'^<fc. (11) 

Upon substituting Equation 9 into Equation 11 it can be seen that 

c,=l£.(Oe^^* = is(-y^), (12) 

from which Equation 8 follows when Equation 12 is substituted back into 
Equation 10. 



If the function. s(t)€ ^ is used in Equation 8, instead of the 
function s(t), one obtains the result 



I s(t^nT)e-'- sU^^^)e''~ . (13) 

By setting tsOin Equation 13 one arrives at the familiar sampling equation 
S(e'-) = ^tsU^), (14) 

which appears as Equation 3.20 in A. V. Oppenheim and R. W. Schafer, 
"Discrete-Time Signal Processing," Prentice Hall, 1989. Equation 14 indicates 
that the discrete Fourier transform of a sampled analog signal is comprised of 
periodically repeated copies of the continuous Fourier transform of the analog 
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Signal. The copies are shifted by integer multiples of the sampling frequency and 
superimposed. Equation 14 is often expressed in terms of delta functions; namely, 
that the Fourier transform of an impulse train f^Sit-nT) is itself an impulse 

It is noted that Equation 14 can also be derived directly from 
Equation 7 by using s(t)e''^ instead of s(t). 

For values of oi that are small enough to avoid significant aliasing 
in Equation 14, say ax Q„ one can approximate the sums on the right hand sides 
of Equations 13 and 14 by the term with * = 0. When this approximation is made, 
it in turn leads to the approximation 

ts(t + nT)e-'-.S(ene'^ , H<fl,. (15) 

It is noted tiiat as the sampling period, T, decreases, the approximation in 
Equation 15 becomes more accurate. The increase in accuracy is due to the feet 
that Uie aliasing, or overlap, between the copies on the right hand sides of 
Equations 13 and 14 decreases, since the shifts ~ between copies increase. 

The present invention uses a version of Equation 15 to obtain 
super-resolution. Specifically, instead of sampling s[n} = s(nT) as in Equation 5, 
the sampling is done through an integral 

[^s(t + nT)g(t)dt , (16) 

where g(t) represents the sensitivity, or apodization, of a pixel. Multiplying both 
sides of Equation 15 by g(t) and integrating, results in 



7 (17) 



which is the form used in the present invention. 

Referring back to Equation 3. it follows from Equation 17. upon 



replacing r by ^.that 

AT 



Oi-j--—) 
Ax 



Ax 



<n„ (18) 



where U(ei'") is die discrete Fourier transform of die samples u(n— ) . sought to 

K 

be reconstructed; namely, 
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C/(e^"')=I«(«fK- . (19) 
The Fourier transfonn is known from the samples yfnj in Equation 3, and 
the Fourier transform G(-y^)is determined from knowledge of the sensitivity 

function g(x). It is noted that as ^ increases, the approximation in Equation 18 
becomes more accurate. 

Regarding the Fourier transform 7(61"), the values of y[n] set 
forth in Equations above are only defined for indices n = 0, 1, KN-1. For 
indices n outside of this range, y[n] is preferably defined to be zero. With such a 
convention, the Fourier transform YieJ") is given by the finite sum 

KV-I 

n^'*')=Z>'H^""" • (20) 

In a preferred embodiment, the present invention uses Equation 

18 to determine the Fourier transform Uiei"^), and then reconstructs the values 
Ax 

"C'*']^) r'^ - 0, 1, from LYe/**) by using an inverse discrete Fourier 

transform. Preferably a Fast Fourier Transform (FFT), as is familiar to those 
skilled in the art, is used to invert the spectral values U(dl'^). The reconstruction is 
approxhnate, since Equation 18 is itself only an approximation. Since it is 
necessary to reconstruct KN values of u(x), it is accordingly necessary to apply 

Equation 18 at KN frequencies that are spaced — apart one from the next, since 

KN 

the Fourier transform U(ei'") is periodic with period 2fr. In a preferred 
embodiment of the present invention, Equation 18 is applied at the KN frequencies 

2;dc 

= — . k = 0,1,..., KN-1. (21) 
When the Fourier transform IKef") is inverted it is possible that 
values of «(«^) are non-zero outside of the index range n = 0,I. .... KN- 1. As 
• • A« 

U IS only the values of «(/»—) within the index range n = 0, 1, ..., KN- 1 that are 

of interest, the values outside this index range can be ignored. 

An apparent difficulty arises in implementing Equation 18 if ±ere 

are frequencies m as given in Equation 21, at which G(-y— ) = 0, since this 

Ax 

would entail division by zero at such frequencies. 

The function G(-jQ is typically a real-valued function, since the 
pixel sensitivity function g(x) is typically symmetric about x^O. Since a pixel is 
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typically more sensitive to light in its central area and less sensitive in its outer 
regions, the pixel sensitivity function g(x) is typically "bell-shaped," similar to 
Gaussian functions. In case g(x) happens to be a Gaussian function, its Fourier 
transform has no zeros, and there no divisions by zero are encountered in 
implementing Equation 18. However, for other sensitivity functions, such a 
difGculty can arise. 

The present invention avoids such a difBcuIty by using specially 
constructed masks, as described hereinbelow. 

Masking 

In order to avoid division by zero in Equation 18, the present 

invention preferably employs an optical device to remove zeros of (?(-/—) that 

Ax 

are less than the frequency /3^. TTie frequency A is the maximal frequency in the 
reconstructed data. 

The present invention preferably attaches a mask in the form of a 
fine transmission grating with a period of 4c to the sensing device panel. As 
mentioned hereinabove, the mask under discussion is really a two-dimensional 
doubly periodic mask that covers a two-dimensional array of detectors. For the 
sake of clarity and definiteness it is being described as a single dimensional 
periodic mask, with the understanding that the one-dimensional analysis presented 
herein appUes to each of the two dimensions of the sensing device panel. 

Reference is now made to Figure 7, which is a simplified 
iUustration of a mask attached to a sensor panel of array detectors in an image 
sensing device, in accordance with a preferred embodiment of the present 
invention. A sensor panel 700 in an image sensing device contains a two- 
dimensional array of detectors 710. Detectors 710 are depicted as individual ceUs 
within sensor panel 700. The array of detectors is periodic, with periodic spacings 
4x and /ly in the horizontal and vertical dimensions, respectively. The spacings /ix 
and correspond to the pixel spacings in the sampled digital image obtained 
from the image sensing device. A masking pattern 720 in the form of a fine sub- 
pixel transmission grating is used to generate a doubly periodic mask, with 
respective periods zl* and zly in the horizontal and vertical dimensions. In a 
preferred embodiment of the present invention, the doubly periodic mask is 
attached to sensor panel 700. Each detector 710 is masked by the same masking 
pattern 720. 
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Use of a mask decreases the effective size of each pixel so that the 
Fourier transform G(jQ) of its sensitivity function has a wider band, and its zeros 
accordingly move to higher frequencies. The apodized pixel, determined by the 
transmission pattern of the grating, is preferably designed in an iterative manner, 
as described hereinbelow, so that the bandwidth of the Fourier transform GQO) of 
its sensitivity function is wider than the maximal frequency to be reconstructed. 
Use of a transmission grating has the disadvantage of decreasing the energy sensed 
at each pixel, since it blocks out some of the illumination. 

A mask is described mathematically by a spatial function m(x), 
preferably taking values 0 and 1. The effect of a mask is to modify the pixel 
sensitivity function from g(x) to p(x) « g(x)m(x). An objective of choosing a 
suitable mask is to obtain a function p(x) whose Fourier transform does not have 
zeroes at frequencies that are between 0 and /2y, so that when the Fourier 

transform ^(""J "^) modified pixel sensitivity function p(x) is used in 

Equation 18 instead of G(- ; — ) , there is no division by zero. 

A rationale for using a mask to avoid zeroes of G is that typically 
an effect of a mask is to add frequencies that may have not been present 
beforehand. 

To construct a mask as desired, the present invention preferably 
uses a grating in the form of a plurality of optically transparent slits. 

mix)^l-f^rect(^)®S(x-x,), (22) 

where M is the total number of slits, xj, x2, ...,xm are the locations of the slits and 
&l is the width of the /-th slit. The function rectQ is given by 



_ -l/2<z<l/2, 
rectiz)^\^ . (23) 

otherwise 



and ® denotes convolution. 

Reference is now made to Figure 8, which illustrates the shape of 
a mask in accordance with Equation 22. Such a mask consists of M slits 810, 
centered at positions xj, X2, x/i/, each having respective widths of &/. 
Preferably,' the value of M is a number between zero and and the values &/ are 
each approximately equal to the sub-pixel size Ax I K. As can be seen in Figure 8, 
the values of the mask m(x) are 0 inside the slits and 1 outside the slits. The total 
width of the mask if Ax\ namely, the width of a pixel. The mask is intended to 
affect the shape of g(x) at sub-pixel granularity. In practice, the granularity is 
discretized to Ax I K; namely, the sub-pixel shift between successive images 
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acquired by the image sensing device. For example, referring back to Figure 3C, 
the mask used in reconstructing the image shown therein is [0 1 0 1 1 0 0 0], each 
of the eight entries in this array representing a sub-pixel size of one eighth of a 
pixel. Thus the function m(x) for the mask used in Figure 3C is given for 
0<.r<Axby 

lo ^ 8 8 8 • (24) 

^ ' otherwise 

The sensitivity function of each masked pixel is given by 
p(x) ^ g(x)m(x) , (25) 
and it can be seen that the Fourier transform of p(x) is given by 

= G(;a)-|;&,G(/Q)®[e-'^^sinc(iQ&,)] , (26) 
(•I 2 

where sinc(z) denotes the function sinc(z) = . 

z 

Preferably, to determine an optimal mask, a predetermined value 
of M is selected, and a search is made for position values xi, X2, W ^nd 
widths &CJ, Sx2, &cj^ for which the |P(;n)| is bounded away from zero as 

much as possible. Specifically, define 
(t(;c,,^2,...,a:^ ;&i,&2,..,,&^) = 

==mn|P(;n;jc^,;c2,:..,x^;&i,<&2,.,-,&t^ , (27) 

where i?^ denotes the frequency up to which no significant aliasing occurs, as 
defined above. Then optimal values for xj, X2, xm and Sxj, &2, &Af are 
preferably determined by 

= arg-max(7(jc,,A;2,...,^^;<&j,<&2v..,&w) » (28) 
where arg-max denotes the arguments xj, X2, xm and Sxj, 3x2, <Sca/ which 
maximize the function cr. 

The search range over which optimal values ofxj, X2, xm and 
&/, 5x2, are sought is such that the total + &2 + + of ^1 ^be 

widths does not exceed the pixel width Additionally, each individual width 
&i must not be less than 2a, where X is the wavelength of the incoming light - 
otherwise, the light will not be affected by the mask. Moreover, construction of a 
mask and attaching it to the detectors may further limit the widths &i to exceed a 
minimum value of 

From energy considerations, it is clear that on the one hand, it is 
desirable that each width &i be as small as possible, since Sxi is proportional to 
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the energy blocked by the mask. On the other hand, this must be traded off against 
the condition that \PijQ)\ be bounded away from zero. 

Specifically, if denotes the total light energy for a typical 
viewing target, when viewed without a mask, then the energy detected when a 
mask is present is given by 

^f 

^ = ^.(1-^). (29) 

since the energy blocked by the slits of the mask is proportional to the total width 
of the slits combined. 

Thus for a given minimum energy E„un^ required to activate the detectors, the 
widths &i are constrained to satisfy 

X<&,^a-^)Ax. (30) 

M Ef 

Condition (30), together with the condition that Sc, i for i = /, 2, M, 
determine the search range for widths &/, (&2^ &cm over which a maximum of 
aixj, xz xm; &C2, &2, dx^) is sought. 

When values of xj, xz xm and dxz &M are 
detennined according to Equation 28, Equation 22 is used to generate an optimal 
mask. 

In and alternative embodiment of the present invention, rather 
than maximize o(xi x^ xm; &h Skz -v &M)f the values ofxj, xz xm 
and &j, dxz Sxm are determined by maximizing a weighted function o(xi, xz 
xm; Sxi 6x2, "f 3xm) M&i, 5x2, Sxm)^ where w is an appropriate weight 
function. For example, w can put higher weight on smaller values of the widths 
<&i, so that preference is given to masks having slits with smaller widths. 

Reference is now made to Figures 9A - 9J, which illustrate the 
use of a mask in obtaining super-resolution, in accordance with a preferred 
embodiment of the present invention. Figure 9A illustrates an input object, such 
as a barcode, represented as a function u(x). Figure 9B illustrates the Fourier 
transform of u(x). Figure 9C illustrates a pixel sensitivity function g(x). Figure 
9D illustrates the Fourier transform of g(x). 

Figure 9E illustrates conventional sampling of u(x) as a digital 
signal, without using super resolution enhancement. Figure 9F illustrates the 
Fourier transform of the digital signal of Figure 9E, Figure 9G illustrates 
sampling of u(x) as a digital signal with a resolution eight times higher than the 
sampling illustrated in Figure 9E. Figure 9H illustrates the Fourier transform of 
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the digital signal of Figure 9G. Figure 91 Ulustrates a mask used in accordance 
with a preferred embodiment of the present invention. 

Figure 9J illustrates the Fourier transform of a masked pixel, 
using the mask of Figure 91. By comparing Figure 9D with Figure 9J it can be' 
seen that the spectrum of the masked pixel does not contain zeros within the 
region displayed. It can further be seen that an effect of the masking is to expand 
the spectrum ofg(x). 

Reference is now made to Figure 10, which illustrates an 
enhanced image obtained by applying super-rcsolution to the low resolution CCD 
captured image iUustraied in Figure 3B, without the use of a mask. As can be seen 
in Figure 10, artifacts in the form of vertically oriented white stripes arise due to 
the presence of zeroes of G(-j^). When a mask is applied in accordance with 

a preferred embodiment of the present invention, Uie zeroes of ) are 

Ax 

removed, and the result obtained is tiie image shown in Figure 3C. It may thus be 
appreciated diat the use of a mask serves to reinove the artifacts indicated in 
Figure 10. 

Imnrnvgri Apppiyf^oH»» 

As explained hereinabove, Uie approximation in Equation 18 is 
based on the assumption tiiat co is small enough to avoid aliasing in Equation 14. 
A more accurate approximation can be made by taking into consideration die 
aliasing between adjacent copies of 5 in die right-hand side of Equation 14. 
However, in order to obtain tiie increase in accuracy, it is necessary to increase tiie 
. sampling rate. 

It is appreciated by those skilled in tiie art Uiat die assumption tiiat 
aliasing between copies of S in Equation 14 is due to overlap between adjacent 
terms 5(;^) and 5(y ^^^ )is equivalent to tiie assumption tiiat reduction of 

the sampling period from T to T/2 removes aliasing. Moreover, tiie information at 
frequency co in the discrete Fourier transform of a signal sampled witii sampling 
period r corresponds wiUi the information at both frequencies a/2 and a}/2 + ;ria 
die discrete Fourier transform of tiie signal sampled witii sampling period T/2. A 
precise foraiulation of tiiis relationship appears in tiie above referenced 
Oppenheim and Schafer as Equation 3.76, which describes a frequency-domain 
relationship between tiie input and tiie output of a sampling rate compressor 
("downsampling"). 
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These observations above are a basis for an improved 
approximation used in accordance with a preferred embodiment of the present 
invention, as described hereinbeiow. Specifically, replace the sampling period T 
by r/2 in Equation 16: 

3'W=£^(/ + /i|)^(0tfr . (31) 
Under the assumption that aliasing in Equation 14 is due to overlap between 
adjacent terms SUf) and SU^),rht approximation toS(d'") becomes 

Sie-)^'^SU^).ISU^), 0<o<., (32) 
for positive values of a>, and 

^{e )-^5(;^) + -50— , -;r<a;<0, (33) 

for negative values of 

Using similar analysis to that embodied in Equations 13 - 18, it 
can be shown that 

^ Ax 

O<0<^, (34) 

for positive values of o), and 



^ 2 



-/r<<o<0, (35) 

for negative values of a>. Equations 34 and 35 determine U(ei'') more accurately 

than Equation 18, since diey incorporate aliasing between adjacent copies of 5 in 

the sampling representation for S(ei'"). However, they require sub-pixel shifts of 
Ax 

~ between successively captured images. This corresponds to a factor of 2K: 

increase in data acquisition, whereas the resolution is only enhanced by a factor of 
K. 

An alternative approach to improving the approximation in 
Equation 18 by taking into consideration Uie aliasihg between adjacent copies of 5 
in the right-hand side of Equation 14, uses an alternating ("checkerboard") mask 
geometry. As mentioned hereinabove, Uie above discussion is based on use of a 
periodic mask, wiUi a period of Ax. By using instead a mask wiUi a period of 24c, 
one effectively generates two pixel sensitivity functions, gj(x) and g2(x), where 
gl(x) applies to even-numbered detectors and g2(x) applies to odd-numbered 
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detectors. Specifically. gi(x) corresponds to g(x)m(x) and g2(x) corresponds to 
8(x)m(x+Axh where g(x) is the pixel sensitivity function for the image sensing 
device and m(x) is the overall mask. Unlike the above case where the mask has a 
period of Ax, when the mask has a period of 2Ax there are effectively two masks 
operating (namely, the mask itself and the mask shifted by Ax), and consequenUy 
the even and odd-numbered pixels are processed differently. 

Reference is now made to Figure 11, which is a simplified 
illustration of an alternating ("checkerboard") mask attached to a sensor panel of 
array detectors in an image sensing device, in accordance with a preferred 
embodiment of the present invention. A sensor panel 1100 in an image sensing 
device contains a two-dimensional array of detectors 1110. Detectors 1110 are 
depicted as individual ceUs within sensor panel 1100. The array of detectors is 
periodic, with periodic spacings 4c and in the horizontal and vertical 
dimensions, respectively. The spacings Ax and Ay correspond to the pixel 
spacings in the sampled digital image obtained from the image sensing device. 

In distinction to Figure 7, Figure 11 illustrates two different 
masking patterns 1120 and 1130 in the form Of fine sub-pixel transmission 
gratings that are arranged in an alternating checkerboard pattern to generate a 
doubly periodic mask, with respective periods 24c and 2dy in the horizontal and 
vertical dimensions. In a preferred embodiment of the present invention, the 
doubly periodic mask is attached to sensor panel 1100. The detectors 1110 are 
alternatively masked by the different masking patterns 1120 and 1130. 

When using such a mask. Equation 2 (which is one-dimensional) 
separates into two equations, 



for even values of n; k^O.l, K-1, and (36) 

" A 

«* [«] = /"(x + nAx + * —■)g, (x)dx , 

for odd values of «; k = 0,l, K-l. (37) 
By capturing twice as many images as previously, namely, 2K images, each 
detector moves over a full period of the mask, namely, 2Ax. Consequently, two 
sequences of the form of Equation 3 are acquired, namely, 

y,H = ju(x'^n—)g(x)dx , rt = 0, i, KN-1 , and (38) 

yzi"] = J«(* + n ~)g(x)<& , n«0,I, KN-1 . (39) 
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In turn, using the same analysis embodied in Equations 13 - 18, one arrives at the 
two conditions 

r.W')'^saf)G,HfH^su^yG,(-jS:^^ , (4«) 

for positive values of 0 < £y < ;r, and 

>^(»*)'i5(;f)G,(-;£).is(;£±2£,c,(-/5i^) , («, 
n(e")450f)C,(-y£).is(y£i?£)c,(-;£^) . (43) 
for negative values ofoi,-K<a)<0. These equations can be used to solve for the 
individual copies SUj) and S(y^y^), and then to recover S(d'') from 
Equations 32 and 33. The final result obtained, expressed in terras of U(ei'^, is 



4« ^ — Ax 



(44) 



for positive values of <y, < a; < a; and 



(45) 



for negative values of cd, -7t< a < 0. In Equations 44 and 45, the terms S>fG('m 
denote the difference 

s^(.m = G(-m - (?r-;Y.f^ . (46) 

In distinction to the approach using Equations 34 and 35, whereby 
the improvement in accuracy stems from reducing the sub-pixel sampling shift in 
u If £_ A t A y 

half, from — to t^ie approach using Equations 42 and 43 obtains 

improvement in accuracy by acquiring 2^ sampled images, each shifted by — . 

K. 

Both approaches require collection of sampled data that is 2A: as much data as in a 
single image captured by a sensing device. They differ in the types of masks used, 
and in the relative shifts between the multiple images acquired. 

When implemented with full two>dimensional processing, both 
approaches described hereinabove require 4K^ as much data as in a single image 
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captured by a sensing device, in order to enhance image resolution by a factor of 
two in each pixel dimension. 

Ggomgtric Simer Resolution v<. piffrartivp Siiper Resnlnrinn 

The multiple image acquisition described above and illustrated in 
Figure 6 is geometric in nature; i.e., it is based on viewing the same spatial 
information several times, such that in each view the object is shifted by a sub- 
pixel amount. It can be implemented by shifting positions of the sensing device 
("temporal multiplexing") or. equivalenfly, by shifting fields of view ("spatial 
multiplexing") using an optical element that repUcates views of the object with 
sub-pixel shifts between the replicas. The resolution of a system limited by 
geometry corresponds to the detection sampling system; i.e., the resolution of 
system sensors. 

In distinction, multiple image acquisition can also be dif&active 
in nature. The resolution of a system limited by diffraction corresponds to the 
finest detail that can pass through the system without being distorted, and is 
proportional to the size of the aperture in the optical system. Thus the resolution 
of the human visual system is limited by the extent to which the eyes are open. 
When an observer squints his eyes, the resolution of a scene being viewed 
decreases, since the size of the opening is decreased. 

For exanq)le, resolution enhancement can be iiiq>lemented by an 
assembly of two moving rotated gratings, a first grating attached to the object 
being captured, and a second grating attached to a sensor of the sensing device, 
and moving in a direction opposite to that of the first grating. Such an assembly 
can be used to enhance the resolution of a diffractive system, since it effectively 
increases the size of the aperture in the optical system. The movement of the 
grating attached to the object encodes its spatial information by a Doppler-Iike 
effect, and allows the information to be transmitted through a restricted aperture of 
an imaging lens. The decoding of the information is obtained &om the second 
grating attached to the sensing device, which moves in a direction opposite to that 
of the first grating. Such an assembly of two rotated gratings is practical for 
microscopic applications. 

However, in many imaging systems the object is distant from the 
observer, and attachment of a grating to the object is impractical. When applied to 
scanning systems such as barcode readers, the present invention preferably 
simulates attachment of a moving grating to the object by illuminating die object 
with an illumination having a spatial structure of a grating. The illumination is 

23 



wo 00/59206 PCTALOO/00201 
Shifted with time in order to simulate temporal motion of the grating. A speciaHy 
designed diffractive optical element is attached to tiie Ulummation source so Uiat a 
grating pattern appears in the object plane. The grating pattern is moved by phase 
modulating the light source, since linear phase modulations appear as shifts in a 
far field. 



Addition al Considerations 

In reading Uie above description, persons skilled in the art will 
realize that there are many apparent variations that can be applied to the metiiods 
and systems described. For example, tiie integral in Equation 3 may be replaced 
by a discrete Gabor transform over u with window function g; namely, 

y[^]-Zs(PY>((^''P)f) ' (47) 

Similarly, one may use instead a wavelet or Mellin transform, as is well known to 
persons skilled in the art. 

Additionally, the shifts between the multiple images that are 
acquired by the sensing device are not required to be equal. Furthermore, the 
shifts between the multiple images are not required to be horizontally or vertically 
disposed, nor are they required to be disposed in a fixed direction. Wavelet 
transforms and Mellin transforms are particularly advantageous when dealing with 
unequal shifts between acquired images. 

Additionally, the reconstruction algorithm used in the present 
invention, for generating a high resolution image from multiple low resolution 
images, need not be separable. That is, it need not be comprised of horizontal and 
vertical reconstructions. Rather, the reconstruction algorithm used in the present 
invention can be a genuine two-dimensional algorithm. 

In the foregoing specification, the invention has been described 
with reference to specific exemplary embodiments thereof. It will, however, be 
evident that various modifications and changes may be made to the specific 
exemplary embodiments without departing from the broader spirit and scope of 
the invention as set forth in the appended claims. Accordingly, the specification 
and drawings are to be regarded in an illustrative rather than a restrictive sense. 
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CLAIMS 

What is claimed is: 



A method for enhancing the resolution of an image sensing 
device, comprising the steps of: 

attaching a mask to a panel of detectors in an image sensing 

device; 

generating multiple fields of view, the multiple fields of view 
being related to one another by sub-pixel shifts; 

acquiring multiple images with the image sensing device from the 
multiple fields of view; and 

combining the multiple images into an enhanced image of higher 
pixel resolution than the pixel resolutions of the multiple unagcs. 

2- The method of claim 1 wherein the image sensing device is a 

scanner. 



3« The method of claun 2 wherein the image sensing device is a 

barcode reader. 



The method of daim 1 wherein the image sensing device is a 

CCD camera. 



5' The method of claim 1 wherein the mask is a fine transmission 

grating. 

6' The method of claim 5 wherein the panel of detectors comprises a 

periodic array of detectors, and the mask is periodic with period equal to that of 
the periodic array of detectors. 

7- The method of claim 5 wherein the panel of detectors comprises a 

periodic array of detectors, and the mask is periodic with period twice that of the 
periodic array of detectors. 
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8- The method of claim 5 further comprising the step of determining 

the number of slits, the positions of the slits and the widths of the slits in the fine 
transmission grating. 

9. The method of claim 8 wherein said determining step determines 
the number of slits, the positions of the slits and the widths of the slits, so that the 
mask eliminates zeroes in a Fourier transform of a function representing a 
sensitivity of a pixel, within a prescribed range of frequencies. 

10. The method of claim 9 wherein the widths of the slits are 
constrained so as not to block more than a prescribed fraction of light energy from 
reaching the detectors. 

11. The method of claim 9 wherein the widths of the slits are 
constrained so as not to be smaller than a prescribed minimum width. 

12. The method of claim 9 wherein said determining step uses a 
weight function specifying relative weights to be placed on values for the widths 
of the slits. 

13. The method of claim 9 wherein the mask is of the form 

m(x)^l-f^rect(^)®S(x-x,) . 

14. The method of claim 1 wherein said generating step uses natural 
vibrations of a platform supporting the sensing device. 

15. The method of claim 1 wherein said generating step uses at least 
one mirror to shift fields of view of the image sensing device by sub-pixel shifts. 

16. The method of claim 1 wherein said combining step is applied 
separately to rows and columns of pixel data. 
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approximation 
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The method of claim 16 wherein said combining step uses the 



to detennine the rows and columns of pixel data of the enhanced image. 



18- The method of claim 17 wherein said combining step comprises 

applying an inverse Fourier transfonn to values of U{e3''). 



19. 

approximation 
2 



The method of claim 16 wherein said combining step uses the 



for positive values of a, and 



0 <a><;r , 



2 



ne'^)/C7(-;^)*r(e'-f")/G(-/l21i2£}£) 



tx 



-7F<<0<0, 

for negative values of to, to detennine the rows and columns of pixel data of the 
enhanced image. 



20. The method of claim 19 wherein said combining step comprises 

applying an inverse Fourier transform to values of U(el'^. 



21. The method of claim 16 wherein said combining step uses the 

approximation 

Ax' 



Ax 



for positive values of tz;^ 0 < a>< and 



Uie'") « 



Ax ^ —ST ^ 
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c.(-yf,^^<;.(-y^)-c,(-;f)._^G,(-yf)l 

for negative values of ^rc < ax 0, and wherein the terms S^i^jQ denote the 
difference 

to determine the rows and columns of pixel data of the enhanced image. 

22- The method of claim 21 wherein said combining step comprises 

applying an inverse Fourier transform to values of U(d^). 



23- The method of claim 1 wherein said combining step uses a Gabor 

transform. 



24. The method of claim 1 wherein said combining step uses a 

wavelet transform. 



25- The method of claim 1 wherein said combining step uses a Mellin 

transform. 



26- A system for enhancing the resolution of an image sensing 

device, comprising: 

an image sensing device comprising a panel of detectors; 

a mask attached to said panel of detectors; 

a motion generator generating multiple fields of view, the 
multiple fields of view being related to one another by sub-pixel shifts; 

image acquisition circuitry housed within said image sensing 
device acquiring multiple images from the multiple fields of view; and 

a combiner combining the multiple images into an enhanced 
image of higher pLxel resolution than the pixel resolutions of the multiple images. 

27. The system of claim 26 wherein said image sensing device is a 

scanner. 
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2S. The system of claim 27 wherein said image sensing device is a 

barcode reader. 



29. The system of claim 26 wherein said image sensing device is a 
CCD camera. 

30. The system of claim 26 wherein said mask is a fine transmission 
grating. 

31. The system of claim 30 wherein said panel of deteaors comprises 
a periodic array of detectors, and said mask is periodic with period equal to that of 
said periodic array of detectors. 

32. The system of claim 30 wherein said panel of detectors comprises 
a periodic array of detectors, and said mask is periodic with period twice that of 
said periodic array of detectors. 

33. The system of claim 30 further comprising a grating generator 
determining the number of slits, the positions of the slits and the widths of the slits 
in the fine transmission grating. 

34. The system of claim 33 wherein said grating generator determines 
the number of slits, the positions of the slits and the widths of the slits, so that said 
mask eliminates zeroes in a Fourier transform of a function representing a 
sensitivity of a pixel. 

35. The system of claim 34 wherein the widths of the slits are 
constrained so as not to block more than a prescribed fraction of light energy from 
reaching the detectors. 

36. The system of claim 34 wherein the widths of the slits are 
constrained so as not to be smaller than a prescribed minimum width. 
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The system of claim 34 wherein said grating generator uses a 
weight function specifying relative weights to be placed on values for the widths 
of the slits. 

38. The system of claim 34 wherein said mask is of the form 

'"W = 1-1; rect{~) ^Six'-x,). 

39. The system of claim 26 wherein said motion generator uses 
natural vibrations of a platform supporting the sensing device. 

40- The system of claim 26 wherein said motion generator uses at 
least one mirror to shift fields of view of the image sensing device by sub-pixel 
shifts. 

41- The system of claim 26 wherein said combiner is applied 
separately to rows and columns of pixel data. 

42. The system of claim 41 wherein said combiner uses the 

approximation 



Ax 



to detennine the rows and columns of pixel data of the enhanced image. 

43. The system of claim 42 wherein said combiner applies an inverse 
Fourier transform to values of U(ei^). 

44. The system of claim 41 wherein said combiner uses the 
approximation 



^ 2 



Ax Ax 



Q<Q)<7: , 

for positive values of and 
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2 



for negacive values of o), to determine the rows and columns of pixel data of the 
enhanced image. 



45. The system of claim 44 wherein said combiner comprises 

applymg an inverse Fourier transform to values of (/(ef"). 



46. The system of claim 41 wherein said combiner uses the 

approximation 



for positive values of ;^ and 



for negative values of <n -^r < <d < 0, and wherein the terms S^-J^ denote the 
difference 

SiC(-jn) = G(-j^ - G('j(n- yj; , 

to determine the rows and columns of pixel data of the enhanced image. 



47. The system of claim 46 wherein said combiner comprises 

applying an inverse Fourier transform to values of U(d''^. 



48. 

transform. 



The system of claim 26 wherein said combiner uses a Gabor 



49. 

transform. 



The system of claim 26 wherein said combiner uses a wavelet 
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50. The system of claim 26 wherein said combiner uses a Mellin 

transform. 

A method for enhancing the resolution of an image sensing 



device, comprising the steps of: 

creating replicas of fields of view using an optical element 
attached to an image sensing device; 

acquiring multiple images with the sensing device from the 
replicas of fields of view; and 

combining the multiple images into an enhanced image of higher 
pixel resolution than the pixel resolutions of the multiple images. 

52. The method of claim 51 wherein the optical element is a 
diffractive optical element. 

53. The method of claim 51 wherein the optical element is a 
reflective optical element. 

54. The method of claim 51 wherein the optical element is a 
combined diffractive and reflective optical element. 

55. The method of claim 51 wherein the optical element produces an 
effect of a grating. 

56. The method of claim 51 wherein the optical element is multi- 
faceted. 

57. The method of claim 51 wherein die replicas of fields of views 
are non-overlapping. 

58. A system for enhancing the resolution of an image sensing 
device, comprising: 

an image sensing device; 
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an optical element attached to said image sensing device, said 
optical element being such as to create replicas of fields of view; 

image acquisition circuitry housed within said image sensmg 
device acquiring multiple images from the replicas of fields of view; and 

a combiner combining the multiple images into an enhanced 
image of higher pixel resolution than the pixel resolutions of the multiple images. 

The system of claim 58 wherein said optical element is a 
diffractive optical element. 

60- The system of claim 58 wherein said optical element is a 

reflective optical element. 

The system of claim 58 wherein said optical element is a 
combined diffractive and reflective optical element. 

62. The system of claim 58 wherein said optical element produces an 

effect of a grating. 



63. The system of claim 58 wherein said optical element is multi- 

faceted. 



64. The system of claim 58 wherein the replicas of fields of view are 

non-overlapping. 
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